Saint Barthélemy
MIRAI: Evaluating LLM Agents for Event Forecasting
Ye, Chenchen, Hu, Ziniu, Deng, Yihe, Huang, Zijie, Ma, Mingyu Derek, Zhu, Yanqiao, Wang, Wei
Recent advancements in Large Language Models (LLMs) have empowered LLM agents to autonomously collect world information, over which to conduct reasoning to solve complex problems. Given this capability, increasing interests have been put into employing LLM agents for predicting international events, which can influence decision-making and shape policy development on an international scale. Despite such a growing interest, there is a lack of a rigorous benchmark of LLM agents' forecasting capability and reliability. To address this gap, we introduce MIRAI, a novel benchmark designed to systematically evaluate LLM agents as temporal forecasters in the context of international events. Our benchmark features an agentic environment with tools for accessing an extensive database of historical, structured events and textual news articles. We refine the GDELT event database with careful cleaning and parsing to curate a series of relational prediction tasks with varying forecasting horizons, assessing LLM agents' abilities from short-term to long-term forecasting. We further implement APIs to enable LLM agents to utilize different tools via a code-based interface. In summary, MIRAI comprehensively evaluates the agents' capabilities in three dimensions: 1) autonomously source and integrate critical information from large global databases; 2) write codes using domain-specific APIs and libraries for tool-use; and 3) jointly reason over historical knowledge from diverse formats and time to accurately predict future events. Through comprehensive benchmarking, we aim to establish a reliable framework for assessing the capabilities of LLM agents in forecasting international events, thereby contributing to the development of more accurate and trustworthy models for international relation analysis.
- Asia > North Korea (0.14)
- Oceania > Australia > Australian Indian Ocean Territories > Territory of Cocos (Keeling) Islands (0.14)
- North America > United States > California > Los Angeles County > Los Angeles (0.14)
- (234 more...)
- Law (1.00)
- Government > Foreign Policy (1.00)
- Government > Military (0.93)
- Information Technology (0.92)
Digital Divides in Scene Recognition: Uncovering Socioeconomic Biases in Deep Learning Systems
Greene, Michelle R., Josyula, Mariam, Si, Wentao, Hart, Jennifer A.
Computer-based scene understanding has influenced fields ranging from urban planning to autonomous vehicle performance, yet little is known about how well these technologies work across social differences. We investigate the biases of deep convolutional neural networks (dCNNs) in scene classification, using nearly one million images from global and US sources, including user-submitted home photographs and Airbnb listings. We applied statistical models to quantify the impact of socioeconomic indicators such as family income, Human Development Index (HDI), and demographic factors from public data sources (CIA and US Census) on dCNN performance. Our analyses revealed significant socioeconomic bias, where pretrained dCNNs demonstrated lower classification accuracy, lower classification confidence, and a higher tendency to assign labels that could be offensive when applied to homes (e.g., "ruin", "slum"), especially in images from homes with lower socioeconomic status (SES). This trend is consistent across two datasets of international images and within the diverse economic and racial landscapes of the United States. This research contributes to understanding biases in computer vision, emphasizing the need for more inclusive and representative training datasets. By mitigating the bias in the computer vision pipelines, we can ensure fairer and more equitable outcomes for applied computer vision, including home valuation and smart home security systems. There is urgency in addressing these biases, which can significantly impact critical decisions in urban development and resource allocation. Our findings also motivate the development of AI systems that better understand and serve diverse communities, moving towards technology that equitably benefits all sectors of society.
- North America > United States (0.67)
- Oceania > Samoa (0.04)
- Oceania > Pitcairn (0.04)
- (204 more...)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (1.00)
- Information Technology > Smart Houses & Appliances (0.54)
- Health & Medicine > Public Health (0.48)
- Banking & Finance > Economy (0.46)
- Oceania > Australia > Australian Indian Ocean Territories > Territory of Cocos (Keeling) Islands (0.18)
- Oceania > Samoa (0.08)
- Europe > Netherlands (0.08)
- (228 more...)
Deep Learning Models for Early Detection and Prediction of the spread of Novel Coronavirus (COVID-19)
Ayris, Devante, Horbury, Kye, Williams, Blake, Blackney, Mitchell, See, Celine Shi Hui, Shah, Syed Afaq Ali
SARS-CoV2, which causes coronavirus disease (COVID-19) is continuing to spread globally and has become a pandemic. People have lost their lives due to the virus and the lack of counter measures in place. Given the increasing caseload and uncertainty of spread, there is an urgent need to develop machine learning techniques to predict the spread of COVID-19. Prediction of the spread can allow counter measures and actions to be implemented to mitigate the spread of COVID-19. In this paper, we propose a deep learning technique, called Deep Sequential Prediction Model (DSPM) and machine learning based Non-parametric Regression Model (NRM) to predict the spread of COVID-19. Our proposed models were trained and tested on novel coronavirus 2019 dataset, which contains 19.53 Million confirmed cases of COVID-19. Our proposed models were evaluated by using Mean Absolute Error and compared with baseline method. Our experimental results, both quantitative and qualitative, demonstrate the superior prediction performance of the proposed models.
- Europe > United Kingdom (0.05)
- Europe > Netherlands (0.05)
- South America > Brazil (0.05)
- (250 more...)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Learning Graphical Models > Directed Networks > Bayesian Learning (0.68)
Python Computer Vision Course
Learn Computer Vision. Introduction course to Computer Vision with Python. Make Computer Vision Apps? Learn Computer Vision theory? Build a strong portfolio with Computer Vision & Image Processing Projects? Looking to add Computer Vision algorithms in your current software project ? Whatever be your motivation to learn Computer Vision, I can assure you that you’ve come to the right course. You get. Complete course with 1 hour of video tutorials, Source code for all examples in the course. What you'll learn. Use basic Computer Vision techniques. Do image processing. Build: Image Similarity app, Face Detection app and Object Detection app! Master Computer Vision! .
AI For Marketers: An Introduction and Primer, Second Edition
Keep on file Card Number We do not keep any of your sensitive credit card information on file with us unless you ask us to after this purchase is complete. Your rental will be available for 30 days. Once started, you'll have 72 hours to watch it as much as you'd like! You'll need an account to access this in our app. Please create a password to continue. You agree to our Terms Of Use.
Compositional Questions Do Not Necessitate Multi-hop Reasoning
Min, Sewon, Wallace, Eric, Singh, Sameer, Gardner, Matt, Hajishirzi, Hannaneh, Zettlemoyer, Luke
Multi-hop reading comprehension (RC) questions are challenging because they require reading and reasoning over multiple paragraphs. We argue that it can be difficult to construct large multi-hop RC datasets. For example, even highly compositional questions can be answered with a single hop if they target specific entity types, or the facts needed to answer them are redundant. Our analysis is centered on HotpotQA, where we show that single-hop reasoning can solve much more of the dataset than previously thought. We introduce a single-hop BERT-based RC model that achieves 67 F1---comparable to state-of-the-art multi-hop models. We also design an evaluation setting where humans are not shown all of the necessary paragraphs for the intended multi-hop reasoning but can still answer over 80% of questions. Together with detailed error analysis, these results suggest there should be an increasing focus on the role of evidence in multi-hop reasoning and possibly even a shift towards information retrieval style evaluations with large and diverse evidence collections.
- North America > Guadeloupe (0.05)
- North America > Saint Barthélemy (0.05)
- Europe > United Kingdom > England (0.04)
- (7 more...)
bcr vidcast 107: AI governance, what are AI and ML, and the future is not here yet - Better Communication Results
Vikram Mahidhar reminds us all that AI is only as good as the humans supervising it and programming it. The biases and artefacts that come out of the processing are reflective of the biases programmed in at the beginning. A program trained to recognise totalled car bodies for insurance purposes, for example, will need close supervision of its decision-making outputs, for regulatory and consumer confidence and acceptance of the decision. There is a call and a growth in a new class of AI--one that is explainable, and that builds trust by providing evidence. Vikram also reminds us that a governance strategy is key to engendering trust in our organisation, processes and people.
- Oceania > Samoa (0.06)
- North America > United States (0.06)
- South America > Venezuela (0.05)
- (233 more...)